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Abstract 

This paper considers a parallel system of queues fed by indepen- 
dent arrival streams, where the service rate of each queue depends on 
the number of customers in all of the queues. Necessary and sufficient 
conditions for the stability of the system are derived, based on stochas- 
tic monotonicity and marginal drift properties of multiclass birth and 
death processes. These conditions yield a sharp characterization of 
stability for systems, where the service rate of each queue is decreasing 
in the number of customers in other queues, and has uniform limits 
as the queue lengths tend to infinity. The results are illustrated with 
applications where the stability region may be nonconvex. 
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1 Introduction 

We consider a parallel system of queues fed by independent arrival streams, 
where the service rate of each queue depends on the number of customers in 
all of the queues. This type of model is natural for manufacturing systems 
where a server is capable to process other queues when its own buffer is 
empty, or for cellular radio networks, where the available transmission rate 
for customers in a particular cell is decreasing in the number of customers in 
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the neighboring cehs [2]. Another important category of applications con- 
sists of processor sharing models, where several customer classes simultane- 
ously use one or more servers, whose rate allocations and total processing 
rates may depend on the number of customers in each of the classes [3] . For 
example, in wireless data networks employing channel-aware scheduling, the 
total service rate available to all customers can be increasing in the total 
number of customers, due to multiuser diversity [11]. 

Stability is arguably the most fundamental property of a queueing sys- 
tem, and provides a crude yet useful first-order benchmark of the system 
performance. A general framework for analyzing stochastic stability con- 
sists of Foster-Lyapunov criteria, which are based on finding a suitable test 
function having a positive or negative mean drift in almost all states of 
the state space [7, 14]. In the context of multiclass queueing systems with 
coupled servers, these techniques have been successfully applied to systems 
with utility-based service allocations [21]. Fluid limit analysis is another 
powerful method for finding necessary and sufficient stability conditions for 
a wide class of multiclass queueing networks with work-conserving service 
disciplines [5, 13]. 

The stability analysis of multiclass queueing systems with general state- 
dependent service rates is difficult, because there is no systematic way of 
finding test functions satisfying the Foster-Lyapunov criteria, and the fiuid 
limit techniques are often restricted to systems of work-conserving servers 
with fixed total rate. An alternative means for deriving stability conditions 
is to study whether the system of interest is stochastically comparable to 
a simpler system that is easier to analyze. This approach was first used 
in the multiclass queueing context by Rao and Ephremides [16] and Sz- 
pankowski [18], and later refined by Szpankowski [10], to characterize the 
stability of buffered random access systems. 

In this paper we provide an extension of the above ideas (tentatively 
discussed in [8]), by deriving marginal drift criteria for multiclass birth and 
death processes that allow us to analyze the stability of a broad class of 
parallel queueing systems. Moreover, we present conditions for partial sta- 
bility, where only some of the queues are stable, and give a sharp stability 
characterization for systems, where the service rate of each queue is decreas- 
ing in the number of customers in other queues, and has uniform limits as 
the queue lengths tend to infinity. For systems of at most three queues, 
where the service rates only depend on whether the queues are empty or 
not, our results yield as special cases stability characterizations that have 
earlier been found using transform methods [4, 6] and the ergodic theory of 
deflected random walks [7] . 

The paper is organized as follows. Section 2 describes the model details 
and discusses a notion of stability that is convenient for the subsequent 
analysis. Section 3 presents a key coupling result and marginal drift criteria 
for the stability of multiclass birth and death processes, while the main 
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results regarding the stability of queueing systems are given in Section 4, 
in decreasing level of generality. Section 5 illustrates the results with two 
applications, and Section 6 concludes the paper. 



2 Model description 

2.1 Parallel queueing system with coupled service rates 

We consider a parallel system of N queues, where each queue i is fed 
by an independent Poisson arrival process of rate Aj, and served at rate 
(j)i{Xi, . . . , Xj\f) that depends on the number of customers Xj in each of the 
queues j = 1, . . . ,N. We assume that all customers require independent 
exponentially distributed amounts of service with unit mean, and that the 
system has unlimited buffer space to accommodate customers. The schedul- 
ing at each queue can be first-come first-served, processor sharing, random 
order of service, or any discipline that does not depend on the service re- 
quirements. 

Under these assumptions we can model X = {Xi, . . . , Xjsf) as a continuous- 
time Markov process on with transitions x x + Ci occurring at rate Aj 
and transitions x x — ei > at rate 4'i{x)^ where Cj denotes the z-th unit 
vector in Z^. We assume that the allocation function = ((/>!, . . . , is 
bounded, which guarantees that the process X is nonexplosive. Hence we 
may assume that X and all other stochastic processes treated in the sequel 
have paths in the space D = L>(R_|_, Z;!^) of right-continuous functions from 
M_i_ to Z^ with finite left limits. Recall that a stochastic process with paths 
in D can be viewed as a random element on the measurable space (D,2?), 
where T> denotes the Borel cj-algebra generated by the standard Skorohod 
topology [9]. 

Observe that this model also covers scenarios where the service require- 
ments of customers at queue i are exponentially distributed with parameter 
Hi ^ 1, via replacing 4>i{x) by Hi(t)i{x). 

2.2 Stability notions 

A stochastic process X taking values in a countable space S and having 
paths in ^(M-i-,^) is said to be stable, if for any e > there exists a finite 
set K such that 



and otherwise X is said to be unstable. Further, the process X is called 
transient, if X{t) — )• oo almost surely, that is, for any finite set K, 




for all t 



(2.1) 



V{^C]{X{t)iK}) = l. 



s>Ot>s 
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An alternative way to express (2.1) is to say that the family of distributions 
{P oX{t)~^}t>o is tight. Observe that an irreducible Markov process is 
stable if and only if it is positive recurrent [9, Theorem 12.25]. The following 
proposition illustrates an intuitively clear relation between transience and 
instability. 

Proposition 1. Any transient stochastic process X having paths in £)(M_|_, S) 
is unstable. 

Proof. If X is transient, then for any finite set K there exists an s such that 
V{r\t>s{X{t) i K}) > 1/2. Hence, suptP(X(t) i K) > 1/2 for all finite K, 
so X cannot be stable. □ 

In most applications it is natural to assume that the Markov process 
describing the system is irreducible, in which case stability is equivalent to 
the existence of a unique stationary distribution. In Section 4, where the 
service rates of the original system are modified in various ways, it may 
happen that some of the modified Markov processes are not irreducible. 
This is why we need the following result to guarantee the existence of a 
stationary distribution for a stable multiclass birth and death process under 
slightly weaker than usual assumptions on the reachability of states. We 
denote by X[x\ the version of a Markov process X started in state x. 

Proposition 2. Let X he a N-class birth and death process with strictly 
positive birth rates Aj and bounded death rates (j)i{x). Then the following are 
equivalent: 

(i) X[x\ is stable for some initial state x. 
(a) X[x] is stable for all initial states x. 

(Hi) X has a unique stationary distribution vr supported on a set C such 
that Y'x{X{t) € •) — 7- vr m total variation for all initial states x, and 
X[x\ is irreducible and positive recurrent for any x G C. 

Moreover, X is unstable if and only if X[t) — > oo in probability, regard- 
less of the initial state. 

Proof. See Appendix A. 2. □ 

The following stability characterization of vector-valued stochastic pro- 
cesses is well-known. Because the proof is short, we give it here for com- 
pleteness. 

Proposition 3. A stochastic process X = (Xi, . . . , X^) taking values in a 
countable space Si x • • • x Sn is stable if and only if Xi is stable for each i. 
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Proof. First assume that X is stable. Given e > 0, let us fix a finite set K 
such that supj P{X{t) ^ K) < e, and choose a finite rectangle ETi x • • • x Kn 
that contains K. Then for any i and ah t, P{Xi{t) ^ Ki) < P{X{t) ^ K), 
so it follows that Xi is stable. 

For the reverse direction, it suffices to observe that for an arbitrary finite 
set K = Ki X ■■■ X Kn, P(.X{t) ^ i^) < E» ^ □ 

3 Multiclass birth and death processes 
3.1 Stochastic comparison 

When X and Y are random elements taking values in a partially ordered 
measurable space, we denote X <st Y and say that X is stochastically less 
than Y, if E f{X) < E f{Y) for all positive increasing measurable functions. 
We use the terms increasing and positive in the weak sense, so that a function 
/ is increasing, if /(x) < f{y) for all x < y, and positive if /(x) > for all 
X. Moreover, we denote X =st Y, if the distributions of X and Y are equal. 

Let us endow the spaces and D(M_|_,Z^) with the usual coordinate- 
wise partial orders, so that x < y in if and only if Xi < yi for all z; and 
X <y in L'(M+, Z;!^) if and only if Xi{t) < yi{t) for all i and t. Recall that by 
Strassen's theorem [10, Theorem 1], the stochastic processes X and Y having 
paths in Z)(M_|_,Z^) satisfy X <st Y if and only if there exist processes X 
and Y defined on a common probability space such that X =st X, Y =st Y, 
and Xi{t) < Yi{t) for all i and t almost surely. The following result indicates 
a fundamental relation between stochastic ordering and stability. 

Proposition 4. Let X and Y be stochastic processes with paths in D(IR_|_, Z:!Jf ) 
and assume that X <st Y . 

(i) If X is transient, then so is Y . 

(a) IfY is stable, then so is X. 

Proof. The first claim is a direct consequence of Strassen's theorem, while 
the second follows directly from the definition of stability, because P(|X(t)| > 
r) <V{\Y{t)\> r) ioT al\r a,ndt. □ 

A Markov process having paths in D(R+, "L^) is called a multiclass birth 
and death process, if its transitions are given by 

X ^ X + Ci at rate Xi{x), 

X ^ X — Ci at rate 0j(x)l(xj > 0), 

where \i{x) and (^i(x) are some positive functions on Z^, called the class-i 
birth rates and death rates, respectively. The following lemma, which is 
proved using a direct coupling construction, gives a sufficient condition for 
the comparability of two multiclass birth and death processes. 
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Lemma 1. Let X = {Xi, . . . , Xj) and Y = (Yi, . . . , Yj) be multiclass birth 
and death processes such that X has birth rates Xi{x) and death rates (j)i{x), 
and Y has birth rates r]j{y) and death rates ipj{y)- Assume that for all 
i = 1,...,/ A J, and all x G and y G 7,-^ such that Xi = yi and 
(a;i,...,x/Aj) < (yi, • • • ,2/Mj), 



Ai(x) < r]i{y), 



(3.1) 
(3.2) 



Then for all x € and y € "L"!^ such that (xi, . . . , x/aj) ^ (yii • • • j Ulhj); 



(Xi[x],. 


..,^/ajN) <st {Yi[y],. 






where X[x] andY[y] are 


versions of X and Y started in x and y 


respectively. 


Proof. Let {X,Y) be the Markov process with paths in D{M 
U = {{x,y) G Z^xZ:f^ : (xi, . . . ,x/aj) < ivi,-- • ,2/Mj)}, havin 
transitions 


+ ,U), where 
g the upward 


{x,y) ^ {x + ei,y) 


at rate Aj(x), 


i < / A J, Xi < yi, 


{x,y) ^ {x,y + a) 


at rate r/i(y), 


i < / A J, Xi <yi, 


{x,y) {x + ei,y + Ci) at rate Aj(j;), 


i < I AJ, Xi= yi, 


{x,y) ^ {x,y + a) 


at rate ?7i(y) — Aj(x) 


, i < I A J, Xi = yi. 


{x,y) ^ {x + ei,y) 


at rate Aj(x), 




i>lAJ, 


{x,y) ^ {x,y + a) 


at rate r/i(y), 




i>lAJ, 


and the downward transitions 






(x,y) H> (x - ei,y) 


at rate 4>i{x), 


i<lAJ, 


< Xi< yi, 


(x,y) ^ {x,y - a) 


at rate ipi{y), 


i<lAJ, 


< Xi< yi, 


{x,y) ^ {x - ei,y - Ci) 


at rate tpi{y), 


i<lAJ, 


< Xi = yi. 


{x,y) H> (x - ei,y) 


at rate (f)i{x) - ipi{y), 


i<lAJ, 


< Xi = yi. 


(x,y) (x - ei,y) 


at rate (pi{x), 




i>lAJ, 


(x,y) H> (x,y - Ci) 


at rate tpi{y), 




i>lAJ. 



In Ught of (3.1) and (3.2), we see that all transition rates described above 
are positive. Moreover, because each of the transitions are mappings from U 
into U, we can be assured that the process {X ,Y) exists. 

By studying the marginals of the transition rates, we see that both X 
and Y are Markov, and that their intensity matrices coincide with those of X 
and Y, respectively. Hence, for all x and y such that (x, y) € U, we have 
constructed versions of X[x] and Y[y] on a common probability space such 
that {Xi[x]{t),...,Xj;,j[x]{t)) < {Yi[y]{t),...,Yj^j[y]{t)) for all t almost 
surely. □ 
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3.2 Marginal drift conditions 

In this section we develop necessary and sufficient conditions for the stabihty 
of a multiclass birth and death process, given that each coordinate process 
except one is known to be stable. The following proposition extends the 
classical Neuts' mean drift condition [15], see also Tweedie [2D]. The proof 
follows closely the principles in Section 19 of Meyn and Tweedie [14]. 

Proposition 5. Let X = {Xi, . . . , Xn+i) be an {n + l)-class birth and death 
process with strictly positive birth rates Aj and bounded death rates (t)i{x) such 
that (j)i{x) = <j)i{xi, . . . ,Xn) only depends on the first n input arguments for 
all i = 1, . . . ,n. Assume that: 

(i) The Markov process X" = (Xi, . . . , X^) is stable and has the stationary 
distribution vr. 

(ii) The birth rate of Xn+i satisfies the condition 

Xn+i < < liminf ^ 7r(x"'). 

Then the process X = {X"' , Xn+i) is stable. 

In particular, if (f)n+i{x) = (/>.„+i(x.„+i) only depends on Xn+i, then 

A„+i < liminf 0„+i(x„+i) 

is sufficient for the stability o/X""*""^, regardless of the initial state. 

To prove the above proposition, we use the following lemma: 

Lemma 2. Let {X,Y) be a stochastic process with values in S x where 
S is countable. Assume that 

(i) — 7- vr weakly for some probability distribution vr on S, 

(a) If — > CX3 in probability. 

Then for all bounded f , 

limsupE/(Xt,yt) < V|limsup/(x,y)|^(x). (3.3) 

Proof. Assume first that S is finite, and let f{x) = \\m.s\xpy^^ f{x, y). Then 
for any e > there exists an r such that /(x, y) < f{x) + e for all x and all 
y > r. It follows that 

Ef{Xt,Yt) <E{f{Xt) + e)l{Yt>r) + Ef{Xt,Yt)l{Yt<r) 

= E f{Xt) + e + E[f{Xt,Yt) - f{Xt) - e]l{Yt < r) 
<Ef{Xt) + e + 2\\f\\PiYt<r), 
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where ||/|| = sup^, ^ y)|. By letting t — > oo and recalling that e is 
arbitrary, we see that the claim holds for a finite set S. 

If S is countably infinite, then for any finite set K, the claim holds for the 
function fK{x,y) = f{x,y)l{x £ K). Hence, because fxix) = f{x)l{x E 
K), we get 

hmsup E f{Xt, Yt) < fi^) + 2| 

X 

Thus the claim follows, because we can make tt{K'^) arbitrarily small by 
choosing K large enough. □ 

Proof of Proposition 5. Let us define V{x) = Xn+i and denote the mean 
drift of V with respect to X by 

AV{x) = A„+i - (j)n+lix)l{Xn+l > 0). 

Let us also define Vm{x) = V{x)l{xn+i < M) for M > 0. Then by Kol- 
mogorov's forward equation [9, Theorem 19.6] we have 

f E,. 1:^Vm{X{s)) ds = E, VM{X{t)) - Vm{x). 
Jo 

Because Vm — > V and AVm — >■ Al/ pointwise as M — )■ oo, and because 
IIAVjvf II < A + II0II, we see by applying dominated convergence on the left- 
hand side, and monotone convergence on the right, that 

/ E^AV{X{s))ds = E^V{X{t))-V{x). 
Jo 

Because V is positive, this implies that 

limsupt"^/" E^AV{X{s))ds>0, 

f-i>oo Jo 

and consequently, 

limsupE^ Ay(X(t)) > 0. 

Now assume that the process X is unstable. Then Proposition 2 im- 
plies that X{t) — 7- oo in probability. Because X^ is stable, it follows that 
Xn+i{t) — 7- oo in probability, and we may conclude by virtue of Lemma 2 
that 

y |a- liminf (/>(rE",x„+i) > limsup E^. AF(X(t)) > 0. 



□ 
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To prove the following converse of Proposition 5, we make the addi- 
tional assumption that (pn+i only depends on the first n input arguments. 
Hence {X'"',Xn+i) is a Markov additive process, where the state space of 
the modulating process is countably infinite. For Markov additive pro- 
cesses where the modulating process only takes on finitely many values, the 
following kind of result is well-known [I, Proposition XI. 2. 11]. 

Proposition 6. Let X = {Xi, . . . , Xn+i) be an {n + l)-class birth and death 
process with strictly positive birth rates Aj and bounded death rates <J)i (x) such 
that (j)i{x) = 4>i{x\, . . . ,Xn) only depends on the first n input arguments for 
all i = 1, . . . ,n + 1. Assume that: 

(i) The Markov process X^ = {Xi, . . . , X„) is stable and has the stationary 
distribution vr. 

(a) The birth rate of Xn+i satisfies the condition 

A„+i > Yl (x")7r(2;"). 

Then the process Xn+i is transient, regardless of the initial state. 

Proof. Let us define the free process (X", as the Markov process with 

values in Z" x Z, so that X" = {Xi, . . . is as before, and conditional 
on X" , X^_|_-|^ is a birth and death process on Z with constant birth rate A^+i 
and time- varying state-dependent death rates cpn+iiX^ (t)) . The process 
represented as 

Xl^,{t) = Xl^,{0) + N,{X^+,t) - Nd (^l^(l^n+i{X^s))ds^ , (3.4) 

where Nh and Nd are independent unit-rate Poisson processes, independent 
of X^. The ergodic theorem for positive recurrent Markov processes [9, 
Theorem 20.21] implies that 

t-^ f (t>n+i{X''{s))ds^ Y, </'n+i(x")7r(x") a.s. 

Therefore, dividing both sides of (3.4) by t and taking t — ?■ oo, we see with 
the help of Lemma 8 (Appendix A. 4) that 

\imt-^xf{t) = \n+i- V (x")7r(x") a.s. 

Because the limit is strictly positive, it follows that X,{^-|^(t) — t- oo almost 
surely. 
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Finally, observe that whenever {X^ , Xn+i) and are started 

in the same initial state, Xn+i can be represented in terms of via the 

Skorohod map [17, Theorem D.l] 

Xn+i{t) = xi^,{t) + sM-xi^,{s)]+. 



s<t 



Because X^_^_^{t) — > oo almost surely, the same is true for Xn+i- □ 



4 Stability results for queueing systems 
4.1 General service allocations 

Let us return to the queueing system described in Section 2.1, so from now 
on X = {Xi, . . . , Xjy) describes the queue lengths of the system, and 4>i{x) 
is the service rate for queue i. The following result gives stability conditions 
valid for an arbitrary bounded service allocation (j) = ...,(/> at). Although 
these conditions are not sharp in general, they may provide useful inner and 
outer bounds for stability regions of complex systems that are not easy to 
analyze exactly. 

Theorem 1. Let X = (Xi, . . . , X^) be the queue length process of the 
system with strictly positive arrival rates Xi and bounded service rates (j)i{x). 
Then, regardless of the initial state, the process Xi is stable if 

Xi < liminf inf 4>i{x), (4-1) 

and transient, if 

Xi > limsup sup 4>i{x). (4-2) 
In particular, X is stable if (4-1) holds for all i. 

Proof. Assume that (4.1) holds for some i. By relabeling the queues if 
necessary, we may assume without loss of generality that i = 1. Let Yi 
be the one-class birth and death process with constant birth rate Ai and 
death rates ipi{xi) = mix2,...,XN (f'lix)- Because Ai < liminfa;j_j.oo V'lC^i)) it 
then follows from Proposition 5 that Yi is stable, regardless of the initial 
state. Moreover, because (pii^) ^ "01 (^i) for all x, it follows from Lemma 1 
that <st for any initial state x. Hence, is stable by 

Proposition 4. 

Analogously, if (4.2) holds, then it again suffices to consider i = 1. In 
that case, we let Zi be the one-class birth and death process with birth 
rate Ai and death rates Xii^i) = snp^^^ (pii^) + where e > is such 
that Ai — e is strictly larger than the right-hand side of (4.2). Then Zi 
is irreducible and Ai > limsup^^_^o(3 xi(3;i), so it follows from the classical 
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theory of ordinary birth and death processes [1, Proposition III. 2.1] that Z\ 
is transient. Applying Lemma 1 once more, we see that <st -'^li^;] for 

any x G Z^. Hence, is transient by Proposition 4. 

Finahy, if we assume that (4.1) holds for all i, then all Xi\x\ are stable, 
regardless of the initial state x, hence X is stable by Proposition 3. □ 

4.2 Partially decreasing service allocations 

For the service allocation the following notion of monotonicity is fun- 
damental in comparing multiclass birth and death processes. A function 
4> = (01, . . . , 4>n) from into M.^ is said to be partially decreasing if for 
all i, 

4>i{^) ^ 4>i{y) for all X < y such that x, = yi. (4-3) 
Note that a function (f) = {(pi, . . . , is partially decreasing, if each of 
the coordinate functions (pi is decreasing. Moreover, any function (p = 
{(pi, . . . ,(Pn) such that (pi only depends on Xj, is partially decreasing. Re- 
call also that a continuous-time Markov process X is said to be monotone, 
if the map x i— )■ E^,. f{Xt) is increasing for all t and for any bounded in- 
creasing function /. Using a result of Massey [12, Theorem 5.2], it can be 
checked that a multiclass birth and death process X with constant birth 
rates and bounded state-dependent death rates (pi is monotone if and only 
if (j) is partially decreasing. 

We define as the n-class birth and death process with birth rates Aj 
and death rates i'^(pi given by the lower partial limits 

^"(/)i(xi, . . . ,x„) = lim inf (pi{xi, . . . ,xn)- (4.4) 

The process may intuitively be viewed as a partially saturated version 
of the queue length process X = {Xi, . . . , Xjv), where queues 1, . . . ,n are 
allocated the asymptotically worst-case service rates as Xn+i, ■ ■ ■ ,Xn tend 
to infinity. We also define 

LnXi,...,K-A)= ^>.(^)vr"(x), (4.5) 

if has a unique stationary distribution ir^, and set L^(Ai, . . . , A„,; (/>) = 
otherwise. Thus, the quantity L"(Ai, . . . , A^; </>) can be interpreted as a 
worst-case average service rate dedicated to queue i in a partially saturated 
system where the numbers of customers in queues n + 1, . . . , tend to 
infinity. For notational convenience, we define L^{Xi, . . . , A„,; (p) = i^(pi for 
n = 0. 

Theorem 2. Let X = {Xi, . . . , X]\f) be the queue length process of the sys- 
tem with strictly positive arrival rates Aj and a bounded partially decreasing 
service allocation cp = {(pi, . . . , (Pn)- Assume that there exists an n such that 

\i<L^\\i,...,\i.i;(P) (4.6) 
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for all i = 1, . . . ,n. Then the processes Xi, . . . are stable, regardless of 
the initial state. 

Before proving Theorem 2, we establish some auxiliary results, the latter 
of which will also be used in Section 4.3. 

Lemma 3. Let 1 < n < N . Then for all i and all x, 

i"'~'^(pi{xi, . . .,Xn-i) < liminf r4>i{xi, . . . 

Proof. Fix an a; G Z" and denote a = i^~^(l)i{xi, . . . Then for any 

e > there exists an r such that 

a-e< inf (pi{xi, . . . , Xn-i,yn, ■ ■ ■ ,yN)- 

yn,---yN>r 

Hence, it follows that for all i/n ^ 'f' and for all s > r, 

a-e< inf . . . , . . . , t/at) 

yn+i,---yN>r 

< inf ((>i{xi,...,Xn-l,yn,---,yN)- 

y„+i,...yM>s 

By taking s — > oo, it follows that a — e < £'^(pi{xi, . . . ,Xn-i,yn) for all 
Vn ^ f, so the claim follows because e is arbitrary. 

□ 

Lemma 4. Assume that the function (j) = {(pi, . . . is bounded and par- 
tially decreasing, and assume that Aj are strictly positive and satisfy in- 
equalities (4-6) for i = 1, . . . ,n. Then the n-class birth and death process 
y" = iXi, . . . ,Y^) with birth rates Aj and death rates is stable. 

Proof. To prove the claim for n = 1, let us assume that Ai < Then 
is a one-class birth and death process with birth rate Ai and state-dependent 
death rates £^(j)i{xi). The stability of Yi follows from Proposition 5, because 
by Lemma 3, 

Ai < ^Vi < liminf^Vi(xi). 

To proceed by induction, suppose that the claim is true for n — 1, and 
assume that the inequalities (4.6) hold for i = 1, . . . ,n. Then Y^~^ is stable 
by the induction assumption, and Proposition 2 shows that has a 

unique stationary distribution tt^~^. Let Z" = (Z", . . . , Z") be the n-class 
birth and death process with birth rates Aj and death rates 



...,Xn) 



r(pi{xi, . . . ,Xn), i = n. 
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This choice of rates imphes that the marginal process {Z^, . . . , is 
Markov and coincides with Y^~^ . Now observe that the condition A„ < 
L'^~^{\i, . . . , A„_i; (/)) is equivalent to 

An < X] ^"~Vn(2;i, • • • ,a;„-i)7r""^(a;i, . . . ,a;„_i), 

so using Lemma 3, we see that 

< \limmfipn{xi,...,Xn-i,Xn)>Tr"~^{xi,...,Xn-i). 

^-^ I X„-|-CXD I 

Hence, Proposition 5 shows that is stable. 

Because 4> is partially decreasing, we have £'^~^(j)i{xi, . . . ,Xn-i) < 
for all i < n, and thus i^~^(j)i{xi, . . . , Xn-i) < ^"^^(xi, . . . , x^) for all i < n. 
In particular, it follows that ■i/'iCs^) ^ (■^<t>i{x) for all x and all i < n. Further, 
because i^cj) is also partially decreasing, it follows that i^(j)i{x) > ipidi) for 
all X and y in such that x < y and = Thus, Lemma 1 implies 
that Y"' <st Z"', whenever and Z"" are started in the same initial state. 
Hence, y" is stable by Proposition 4. □ 

Proof of Theorem 2. Assuming the inequalities (4.6) are valid for i = 1, . . . , n, 
we see using Lemma 4 that is stable. Moreover, (j)i{xi, . . . , xn) > 
P^(j)i{xi, . . . , Xn) for alH < n and all x € Z^, because (/> is partially decreas- 
ing. Because t^cj) is also partially decreasing, it follows that (l)i{x) > £"(l)i{y) 
for all X G and y € Z" such that {xi, . . . , x„) < (yi, . . . , y„) and Xj = y,. 
Hence, Lemma 1 shows that . . . , <st {Y{^[y], . . . ,Y^[y]) for 

all initial states x G Z^ and y G Z" such that {xi, . . . , x^) < (yi, . . . , y^)- 
Proposition 4 thus implies that (Xi[x], . . . is stable. □ 



4.3 Partially decreasing service allocations with uniform lim- 
its 

In the following we restrict ourselves to queueing systems where the service 
allocation (p = . . . is such that each coordinate function has a 
uniform limit as some of the input variables tend to infinity. More precisely, 
we say that a function / : "Z^ M has uniform limits at infinity, if the 
following conditions hold: 

(i) There exists a constant such that 

sup \f{x) — f^\ — > 0, as r ^> oo. 

xGX^:xi,...,Xff>r 
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(ii) For any n = 1, . . . , N — 1 and any permutation cr on {1, ... , N}, there 
exists a function /"•'^ : Z" — t- M such that as r — )■ oo, 

sup -r''"(x„(i),...,x^(„))| ^ 0. 

xeZy:x^{„+i),...,x^(jv)>r 

We will next show that the class of functions having uniform limits at 
infinity is rich enough to be of interest. For example, assume that the 
allocation function (j) is of the form 

M^) = gi{xi)h{x) 

where gi has a limit at infinity and h is decreasing. Then gi obviously has 
uniform limits at infinity, and hence the same applies for (pi using the result 
below. 

Proposition 7. Let f and g be bounded functions on Tj^. 

(i) If f is positive and decreasing, then it has uniform limits at infinity. 

(ii) If f and g have uniform limits at infinity, then so do the functions 
f + g and fg. 

Proof. See Appendix A. 3. □ 

If cj) has uniform limits, then the partial lower limits defined in (4.4) 
become true limits in the sense that 

(t){xi, Xn) t'4>{xi, ...,Xn) 

uniformly over xi, . . . , x„, as min(x„_|_i, . . . , xj\j) — )• oo. 

Theorem 3. Let X = {Xi, . . . ,Xn) he the queue length process of the 
system with strictly positive arrival rates Xi and a partially decreasing service 
allocation (j) = . . . ,07v) having uniform limits at infinity. Assume that 
there is an index n such that 

Aj < L*-^(Ai,...,Ai_i;0) foralli<n, (4.7) 

Aj > L"(Ai, . . . , A„; (^) foralli>n. (4.8) 

Then the process (Xn+i, ■ ■ ■ , A'tv) is unstable, regardless of the initial state. 

Proof. Given e > 0, let V^'^ be the n-class birth and death process with 
birth rates Aj and death rates i^4'i{xi, . . . , x^) + e, and let = y"-'0. Then 
is stable by Lemma 4. Further, because i'^cp is partially decreasing, it 
follows by Lemma 1 that Y^'^ <st Y^, whenever Y^'^ and Y^ are started in 
the same state. Hence, Lemma 4 implies that y"'^ is stable for all e > 0, and 
moreover tt"''^ <st tt", where vr"'^ and vr"" denote the stationary distributions 
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of y""'*^ and y"", respectively. In particular, the family {vr"''^}e>o is tight, and 
so by Lemma 6 (Appendix A.l), vr"''^ — >■ tt"" weakly, as e — t- 0. Because (p is 
bounded, it follows that for all i, 

hm V rUx) 7r'^'^(x) = L^{Xi,...,Xn;4>)- 

e->-0 

Hence, by (4.8), we can choose an e > such that 

Ai > ^ t4>^{x) ^"''(x) for all i > n. (4.9) 

Let Z be the A^-class birth and death process with birth rates Aj and 
death rates ipi{xi, . . . , xn) = '^^4>i{xi, . . . ,Xn) + e- Then {Zi,...,Zn) is 
Markov and coincides with Y"'''' as defined above. Moreover, inequality (4.9) 
implies that for all i > n, the mean drift of Zi is strictly positive, so Propo- 
sition 6 implies that Zi is transient for each i > n. 

Next, observe that because has uniform limits, it follows that there 
exists an rg such that il^i{x) > (j)i{x) for all i = 1, . . . , and for all x such 
that Xn+ii ■ ■ ■ jXn > tq. Let us choose an r > tq and define Ar to be the 
complement of the set 

Br = {x £ : Xn+l, ■■■ ,XN >r}, 

Because each of the processes Zn+i, ■ ■ ■ , Zn is transient, it follows that we 
can choose an x G B,. such that 

P,(rA.(Z) = oo) >0, (4.10) 

where ta,XZ) = inf{t > : Z{t) € A^} denotes the hitting time of Z into A^. 
Because cp is partially decreasing, it follows that ^pi{x) > 0i(y) for all x and y 
in Br such that x < y and Xj = yi- Using a similar coupling construction as 
in Lemma 1, it is then straightforward to verify that ryi^(Z[x]) <st T^r(A[3;]) 
for all x G Br- Hence using (4.10) we may conclude that there exists an 
X £ Br such that 

F^{TAr{X) = cx^) > 0. (4.11) 

We now assume that X is stable and derive a contradiction. Using 
Proposition 2, we know that there exists a set C such that X[x] is irreducible 
and positive recurrent for all x € C. Let us now choose an r > tq such that 
C flAr is nonempty. Using Lemma 7 and standard properties of irreducible 
Markov processes [9, Proposition 8.13], we then know that the hitting time 
of X[x] into CCiAr is finite almost surely for all x, which contradicts (4.11). 
Hence, X must be unstable. In particular, because {Xi, . . . ,X„) is stable, 
it follows from Proposition 3 that (A„_|_i, . . . ,Xi^) is unstable. □ 



15 



We are now ready to present our main theorem. For any permutation a 
on {1, . . . ,iV}, we define Af = A„(i) and (^f (x) = ^^(i) (2:^-1 (i), • • • , a;<^-l{^r))• 
The vector A"^ and the function (j)'^ will then correspond to the system where 
queues are relabeled according to a. Denote 

Sn{(I>) = {a e (0, 00)^ : A, < L^HXi, . . . , A,_i; </.) = 1, . . . , iV} , 

where L*~^(Ai, . . . , Aj_i; (p) are given by (4.5), and define 

5(0) = U {A G (0,00)^ : A'^ G SNicPn} , (4.12) 

a 

where the union is taken over all permutations on {1, ... , N}. 

Theorem 4. Assume (p = (0i, . . . ,0Ar) is partially decreasing and has uni- 
form limits at infinity. Then the set S{(p) defined by is open, and 
the queue length process X = (Xi, . . . ,X]\f) of the system with arrival rates 
A = (Ai,...,AAr) and service allocation (p is stable for all A G S{(f>), and 
unstable for all A outside the closure of S{(p). 

Note that it is not possible to characterize the stability of the system 
for arrival rate vectors belonging to the boundary of S{(p) without more 
detailed information on the allocation function (p. Consider for example 
the one-server system where (pi{xi) = (1 + 1/xi)" for some a > 0. Then 
S{(p) = {Ai : Ai < 1}, and the system with Ai = 1 is positive recurrent if 
a > 1 and null recurrent otherwise [1, Section III. 2]. 

Lemma 5. Let (p = {(pi, . . . ,(Pn) be bounded and partially decreasing. Then 
for all n = 1, . . . ,N, the set 

Sn{4>) = {A G (0, 00)^ : Xi < l}-\Xi, Ai_i; 0) Vi = 1, . . . , n} 

is open, and the function = {f^, . . . , /") on (0, 00)^ defined by 

f^{Xu ...,Xn) = I^-\Xi, A,_i; 0) (4.13) 

is continuous on S'„,_i(0). 

Proof. First observe that the function being a constant, is continuous on 
the open set So{(p) = (0, 00)^. To proceed by induction, let us next assume 
that Sn-i{(p) is open and /" is continuous in Sn-i{(p) for some n. To show 
that Sni<p) is open, assume that Sn{(p) is nonempty, and choose a vector 
A G Sn{(p)- Then 

A„ <Lr'(Ai,...,A„_i;0). (4.14) 

Moreover, because Sn{(p) Q Sn~i{(p), it follows that /" is continuous at A, so 
in particular the map rj 1— )• L^~^{rji, . . . , r]n-i', <P) is continuous at A. This im- 
plies that (4.14) remains valid in some open neighborhood Bx of A. Further, 
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because 5'„„i(</>) is open, there is another open neighborhood i?^^ of A such 
that B'^ C Sn-iifp)- It foUows that BxH B'^ CI so we may conclude 

that Sn{(p) is open. 

To complete the induction, we next prove the continuity of /"^^ on 
5„ ((/)), under the assumptions that Sni4>) is open and /" is continuous 
on Sn~i{4>)- If A G Sni4>), then f"- is continuous at A, because Snicp) C 
Sn-i{(p)- Because the first n coordinate functions of f"'~^^ coincide with f"-, 
it is sufficient to prove that the function rj i-)- L'^_^_^{r]i, . . . , rjn, (p) is contin- 
uous at A. Thus, let A^ be a sequence converging to A. Because Sn{(p) is 
open, we may assume without loss of generality that there exists a vector 
A' € Sn{4>) such that A*^ < A' for all k. Let Y be the n-class birth and death 
process with birth rates Aj and death rates i'^(l)i{x), i = l,...,n, and let 

and Y' be the corresponding processes with A replaced by A'^' and A', 
respectively. Then by Lemma 4, all of the processes y, y^ and Y' are 
stable, so we denote their stationary distributions by vr, -k^ , and vr', respec- 
tively. Moreover, because P^(j) is partially decreasing. Lemma 1 shows that 

^st tt' for all /c, so the family {vr^}fc>o is tight. Hence, we can apply 
Lemma 6 (Appendix A.l) to see that tt^ — > vr weakly, so it follows from 
the boundedness of (j) that L'^j^^{\\, . . . , X^; (p) L"_^^(Ai, . . . , A„; (p). This 
shows that f"'~^^ is continuous at A. □ 

Proof of Theorem 4- First, let us note that by using Lemma 5 and relabeling 
the classes if necessary, we see that the set {A : A'^ G Siy{4>'^)} is open for 
all a. Hence, the set S{(f)) is open. Moreover, by again relabeling the classes 
if necessary, we find using Theorem 2 that A G S{(j)) implies the stability of 
the queueing system. 

To study the instability of the system, let us define analogously to (4.12) 
the set 

cr 

where the union is taken over all permutations on {1, . . . , N}, and U]sf{(f)) is 
defined as the set of A G (0,oo)^ such that the inequalities (4.7) and (4.8) 
are valid for some n G {0, . . . , iV — 1}. Then by first relabeling the classes if 
necessary, and then using Theorem 3, we see that the system is instable for 
ah XeU{^). 

It remains to show the instability of the system under the assumption 
that A belongs to the complement of the closure of S{(p) in (0, oo)^, which 
we denote by ext(5). We prove this by showing that for each A G ext(iS) 
there exists a, X gU {(j)) such that A < A and then applying Proposition 4. 

Let A G ext(5). Given a permutation a, let us define n{X,a) as the 
largest integer n such that 

Af < L^HXI, Af_i; (^'^) for all i < n. (4.15) 
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Because A ^ we know that n(A, a) < N for each a, and 

Af >LrHA?,...,Af_i;0-) fori = n(A,a). (4.16) 

Let D be the set of a for which (4.16) is strict, and assume a G D. Then 
by Lemma 5 we see that the function 

(A?, . . . , A^i) ^ {L^.icpn, Lr i(A?, . . . , A^i; ,^'^)) (4.17) 

is continuous in a neighborhood of (AJ^, . . . , A^_^). Thus we can choose 
an €0- S (0,minj Xj) such that the inequahties (4.15) remain vahd and the 
inequahty (4.16) remains strict when A is replaced by A such that ||A — A|| < 

On the other hand, if o" ^ D, then again by the continuity of the function 
in (4.17), there exists an G (0, miujAj) such that the inequalities (4.15) 
remain valid when A is replaced by A such that ||A — A|| < eo-. Moreover, if 
we let A = A — re for some r G (0, Eo-), then it can be checked using Lemma 1 
that L;^-i(Af , . . . , A^_i; cp") < L^-i(Af , . . . , A^.^; (j)"), because the function 
£n-i^a -g partially decreasing, and in particular, is decreasing. Thus 

we see that for n = n(A, a), 

<L:~\x'[,...,Xn-i;r)- 

Let e = miuo- Co- and choose a small enough r € (0, e) such that A = A — re 
belongs to ext(5). Then it follows from the above observations that for all <t, 

Af <Lr'(A?,...,A^i;</.^), i<n{X,a), 
X1^Vr\X'l,...:XU-A''), i = n{X,a). 

Hence, either A^ > L"~^(Af , . . . , A^_]^; t/)"") for all a and all n = n{X,a), 
which implies that A G l^{(t>)', or else, 

n(A, a) > 7i{X, a) for all a, (4.18) 

and X'^ < L^^^lXi, . . . , A^_^; cp'^) for at least some a and n = n{X, a), which 
implies that (4.18) holds strictly for at least one a. 

Assuming A'^ G ext(5) \U{(j)), we can apply the above procedure to A'^ to 
find a A^+^ G ext(5) such that A^+^ < A^, and so that either A'^+i G W((/)), 
or else 

n{X''+\a) > n(A^ a) for all a, (4.19) 

where (4.19) holds strictly for at least one a. The sequence A*^ started in 
A^ = A must hit ^((/>) for some k, because otherwise n{X^,a) = N + 1 
eventually for some k and some a, which would imply that A'^ G 5((/)). 
Hence, X^ ^ U for some k, and A^ < A. Now by Proposition 4, it follows 
that the system is unstable. □ 
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5 Applications 



5.1 Three weakly coupled queues 

Consider a system of three queues where the service rates at each queue 
only depend on whether the other queues are empty or not, so that for all 

O'if Xj — 0, X}^ — 0, 

(j)i{x) = < aij, Xj > 0, Xk = 0, 
1, Xj > 0, Xk > 0. 

Let us assume Oj > aij > 1, so that <j) = {(pi, (j)2, (ps) is partially decreasing. 

Theorem 4 shows that the stability region is a union of six regions corre- 
sponding to the six possible permutations of the queues. The first of these 
regions corresponding to the identity permutation is the set of (Ai,A2,A3) 
such that 

Ai < 1, (5.1) 
A2<Ai + a23(l-Ai), (5.2) 

A3 < flSTTOO + OSlTTlO + a32Vroi + TTii, (5.3) 

where 

TToo = P(ll =0,^2 =0), 
TToi =P(yi =0,^2 > 0), 
7rio = P(yi> 0,^2 = 0), 

TTii = p(yi >o,y2 >o), 

and Y = (Yi, I2) is a random vector distributed according to the stationary 
number of customers in queues 1 and 2 given that the length of queue 3 is 
infinite, which is well-defined when inequalities (5.1) and (5.2) hold. To the 
best of our knowledge, there are no closed-form expressions available for the 
probabilities ttoo, vroi, vrio, vrn, so they must be evaluated numerically. 

The above result coincides with the stability characterization derived 
earlier by Fayolle, Malyshev, and Menshikov using Foster-Lyapunov criteria 
for deflected random walks in Z'^ [7, Theorem 4.4.4]. 

5.2 Two interfering wireless base stations with channel-aware 
scheduling 

We will now turn our attention to a system of two queues served at the 
state-dependent rates 

M^) = gi{xi)hi{x), 

where gi is a bounded increasing function on and hi is a decreasing 
function on . This particular form of allocation function arises as a model 
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for two interfering wireless base stations operating according to a channel- 
aware scheduling discipline, where the functions hi model the interference 
between the base stations, and the functions gi represent the scheduling gain, 
which increases in the number of customers due to multiuser diversity [11]. 
Denote 



g* = lim gi{xi), 








h* = lim sup h 




r— >-oo xi,x2>r 


Xi) = limsup/ij(x). 


3 + i 


Xj~^ca 





and let 



where tt* is the probability distribution (c is a normalization constant) given 
by 



TT^Xi) = cTT —■ , j 7^ i. 

t\g.{z)hlizy 

The stability region can now be described using Theorem 4 as the set of 
(Ai, A2) such that either 

Ai < glhl and A2 < ^a^slAi, 
or A2 < 92^2 and Ai < (A2, </>)• 

Figure 1 represents the full and partial stability regions of the system, 
where the scheduling gains are given by 

gi{xi) = mm{3,log{l + Xi)), (5.4) 

and the interference functions are of the form 

hi(xi) = : , j 7^ i. (5.5) 

The area S is the set of arrival rates such that both queues are stable, 
while the areas Si correspond to the set of arrival rates so that only queue 
i is stable, and U is the set of arrival rates where both queues are unstable. 
Figure 2 illustrates the corresponding stability regions when gi are as in (5.4) 
and ^ 

hiixj) = — ^ — , 7 7^ i. (5.6) 

^ ■" 6 - 4(1 + Xj)-^ ■' ^ ^ ' 
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0.5 \ 1 1.5 0.5 \ 1 1.5 

Ai Ai 



Figure 2: Stability regions for interference functions of the form (5.6) with 
7 = 0.4 (left) and 7 = 2.0 (right). 

6 Conclusion 

We provided sufficient and necessary conditions for the stability of a par- 
allel queueing system with coupled service rates, and showed that these 
conditions are sharp when the service rate at each queue is decreasing in 
the number of customers in other queues, and has uniform limits as the 
queue lengths tend to infinity. Moreover, we presented conditions for par- 
tial stability, where only some of the queues are stable. The most general 
stability conditions, although not sharp, may yield useful inner and outer 
bounds for the stability region of systems that are too complex to charac- 
terize exactly. An interesting and important direction for future research 
is to study whether the given results extend to the case where the service 
allocation does not have uniform limits, and the service times distributions 
are nonexponential. 



21 



Acknowledgements 

The work reported in this paper was carried out while M. Jonckheere and 
L. Leskela were employed by the Centrum Wiskunde &: Informatica, the 
Netherlands. The research has been supported by the Dutch BSIK/BRICKS 
PDC2.1 project. 

A Appendix 

A.l Small perturbations of transition rates 

Lemma 6. Let X and X" he continuous-time Markov processes on a count- 
able state space having transition rates q{x,y) and q'^{x,y), and unique sta- 
tionary distributions vr and vr", respectively. Assume that 

(i) y) — )• y) as n ^ oo for all x and y, 

(a) the set {x : q^{x,y) ^ for some n} is finite for all y, 

(Hi) {7r"}„>o is a tight family of probability measures. 

Then it^{x) tt^x) for all x. 

Proof. Let us assume that tt'^{z) does not converge to tt{z) for some z. Then 
there exists an e > and a subsequence ^ Z+ such that |7r"(z)— 7r(2;)j > e 
for all n € Z^. Because {vr^j^g^Y is tight, there exists a further subsequence 
C Z^ such that tt^ converges weakly to a probability measure vf as 
n — >• oo along Z'_|. [9, Proposition 5.21]. Observe that for all y and for all 7i, 

J]7r"(x)g"(x,y)=0. 

X 

By virtue of assumption (ii), we can take n — )> oo along Z" on both sides of 
the above equation, and bring the limit inside the sum, which shows that 

^Tr{x)q{x,y) = 0. 

X 

Because we assumed the stationary distribution of X to be unique, it follows 
that vf = vr, and hence vr" — )■ vr weakly along Z'^. This is a contradiction, 
because |vr"(2;) — vr(z)| > e for all n € Z^. □ 

A. 2 Stable multiclass birth and death processes with strictly 
positive birth rates 

Lemma 7. Let X be a N-class birth and death process with birth rates Aj 
and bounded death rates (piix), and assume that Aj > for all i. Then the 
hitting time of X into an arbitrary increasing set A is almost surely finite, 
regardless of the initial state. 
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Proof. Let A be an increasing set. Then re ^ A for some positive integer r, 
where e = (1, . . . , 1) € Z^. Let X be the discrete-time jump chain of X, 
with X(n) being the value of X at its n-th jump. Then for all x, 

V,{X{l) = x + ei] 



and because X can reach A from any state x by taking r upward jumps into 
all coordinate directions, we see that P^(X(r^) £A)>5, where 



By induction, it then follows that for all x and all M, 

P^(X(mr^) ^ A Vm = 1, . . . , M) < (1 - 5)*^ 

Thus, by taking M ^ oo, we may conclude that Vx{fA < oo) = 1, which is 
equivalent to Px{ta < oo) = 1. □ 

Proof of Proposition 2. We prove that (i) (ii) {iii), the reverse direc- 
tion being clear. Let us denote x — )■ y if the process X started in x can 
reach y, and let C{x) = {y : x —?■ y and y —?■ x} he the communicating 
class associated with x. Recall that a set C is said to be absorbing if x — > y 
implies y € C for all x € C. Observe first that because all birth rates of X 
are strictly positive, it follows that for all x and y, 

X < y =^ X — > ?/. (A.l) 

From (A.l) we see that all absorbing sets are increasing. Moreover, if a 
communicating class C is increasing, and if x — > ?/ for some x £ C, then 
there exists a z such that x < z and y < z. Hence, y — ?• z by (A.l) and 
z € C, because C is increasing. Because C is a communicating class, it 
follows that y C. We may thus conclude that any communicating class C 
is absorbing if and only if it is increasing. 

We next show that X has a unique absorbing communicating class. As- 
sume first that all communicating classes are nonabsorbing. Then none of 
the communicating classes C(x) is increasing, and Lemma 7 implies that 
Px(tc(x)'' < oo) for all x. Because Fy{Tc(x) = co) = 1 for U ^ C, it 
follows that with probability one, X eventually leaves any finite set without 
ever returning, regardless of the initial state. Thus, X{t) — )> oo almost surely, 
which contradicts the assumption that X started in some initial state, say 
x'', is stable. Hence, X must have at least one absorbing communicating 
class. To see that there is no more than one such class, it suffices to observe 
that if C(x) and C{y) are disjoint sets, then they can not both be increasing. 
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Now let C be the unique absorbing communicating class of X, and as- 
sume that X[x\ is unstable for all x € C. Then for any finite set K and any 
< s < t it follows that 

P,o(X(t) ^K,Tc<s) = Y, ^AX{s) = x,Tc< s) V.,{X{t -s)eK), 

x&C 

(A.2) 

because X{s) belongs to the absorbing set C on the event {tq < s}. Hence, 
by dominated convergence, limj_j.oo Pxo(^(0 ^ K,tc < s) = Q for all s. 
Furthermore, because 

P,o(X(t) G K) < P,o(rc >s)+ P,o(X(t) e K,tc < s), 

and because tq is finite almost surely, we see by taking first t — )■ oo and then 
s — >■ oo that Pxoi-^i't) £ K) — )■ 0, which contradicts the stability of ^[x'']. 
Hence, we may conclude that X[y] is stable for some y € C. Moreover, 
because FxiX{l) = y) > for all x, we see that for all finite sets K, 

limsupP^(X(t + l) eK) > limsupP^(X(l) =y)Py{X{t) e K) > 0, 

SO X[x] is stable for all initial states x € Z^. 

Finally, let X*-" be the Markov process on the state space C with the same 
transition rates as X in C. Then X*^ is irreducible and stable, regardless 
of the initial state. Hence, it follows [9, Theorem 12.25] that X'-^ is positive 
recurrent, and thus has a unique stationary distribution vr*^ on C such that 
the distribution of X'~'{t) converges to vr'^ in total variation. By defining 
7t{B) = TT^ (BnC), it follows that vr is stationary for the unrestricted version 
of X, and because Px{tc < oo) for all x, one can verify using (A.2) that 
the distribution of X{t) converges to vr in total variation, regardless of the 
initial state. 

Having proved the equivalence of (i)-(iii), let now assume that X[x] is 
unstable for all x and show that X(t) — > oo in probability regardless of 
the initial state. We saw above that if all communicating classes of X are 
nonabsorbing, then X{t) is transient, so let us assume that X has the unique 
absorbing class C. Then X[x] is irreducible and positive recurrent for all 
a; E C, so it follows from standard theory [9, Theorem 12.25] that for any 
finite set A, the function hA{x,t) = Px{X{t) £ A) tends to zero as t — > oo 
for all X € C. Denoting the hitting time of X into C by tc, the strong 
Markov property implies that for all x, 

Px{X{t) eA) = P{X{t) (^A,Tc<t)+ P{X{t) £A,Tc> t) 

= Ex hA{X{Tc),t - tc)1(tc <t) + Px(.X{t) eA,Tc> t). 

Because Px{tc < oo) = 1, it then follows from dominated convergence that 
the right-hand side in the above equality converges to zero as t ^ oo. Thus, 
X(t) — > oo in probability. 
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To see that X{t) oo implies the instabihty of X, let us assume that X 
is stable. Then by choosing a finite set A such that vr(^) > 0, we see that 
I'x{X{t) € ^) > for large t. This contradicts the fact that X{t) — > oo in 
probability, so X must be unstable. □ 

A. 3 Uniform limits of monotone functions 

Proof of Proposition 7. (i) We show that when / is decreasing in all its input 
variables, the uniform limits of / are given by 

/0 = inf/(x), 

X 

f"''''{Xa{l),---,Xa(n)) = ^f f {xi, . . . , Xn) ■ 

Observe first that given e > 0, there exists y such that \f{y) — f^\ < e. 
Hence, by defining r = max(?/i, . . . , ?/7v), it follows from the monotonicity 
of / that 

sup |/(x)-/0|<e. (A.3) 

x:xi,...,xisr>r 

This shows that the assertion holds for = 1. 

To proceed by induction, let us assume that the claim holds for all pos- 
itive decreasing functions on Z^"^. Let / be a positive and decreasing 
function on let e > 0, and choose a permutation a. By symmetry, 
we assume without loss of generality that a is the identity permutation, 
and denote /" = /"''^. Using (A.3), we can first choose an ro such that 
— f^\ < e/2 when xi, . . . , x^v > r^. Then by the monotonicity of /, it 
follows that \ f^{xi, . . . , Xn) — /"I < e/2 for all xi, . . . ,Xn > tq. Thus, 

|/(x)-r(xi,...,x„)| <e (A.4) 

for all X such that xi, . . . , xn > r^. 

Let us next choose an z G {1,...,?7,} and yi € {0, ...,ro}. Then the 
function 

(xi, . . . ,Xj_i,Xi+i, . . . ,xn) i-^ f{xi, . . . ,Xi-i,yi,Xi+i, . . . ,xn) 

is decreasing on Hence, by the induction assumption we can choose 

a number ri{yi) such that 

|/(x)- inf /(x)|<e 

Xn+\,...,Xl^ 

for all X such that Xj = yi and x„_|_i, . . . , xat > r{yi). In particular, by 
defining = max(rj(0), . . . , rj(ro)), it follows that 

|/(x)-r(xi,...,xOI <e (A.5) 
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for all X such that Xi < tq and Xn+i, ■ ■ ■ ,X]\f > ri. Finally, by defining 
r = max(ro, ri, . . . , r„), we see by combining (A. 4) and (A. 5) that 

sup |/(x) - P{xi, . . .,Xn)\ < e, 

which completes the induction step. 

(ii) To see that fg has uniform limits at infinity, it suffices to note that 
for any n and any a (omitting the arguments of the functions) , 

l/<7 - /"'V-l < ll/llb - 5"''^! + Iblll/ - /"'I, 

and the same obviously holds for and in place of /"'"^ and g"^'^. Hence, 
the claim for fg follows by taking the supremum over x G such that 
• • • ia;<T(Af) > ?^ on both sides of the above inequality, and then let- 
ting r tend to infinity. The proof for f -\- g is analogous. □ 

A. 4 Strong law of large numbers for a time-changed Poisson 
process 

Lemma 8. Let N be a unit-rate Poisson process, and let A be a stochastic 
process with paths in D{M.^,M^) such that 

lim [ A{s) ds = c a.s. (A.6) 

for some constant c > 0. Then 

\im t-^N (^j^ A{s) ds^ = c a.s. (A.7) 

Proof. We will view N and A as -D(M-|_, M+)-valued measurable mappings 
on a probability space (0,7^, P). Fix a set f^i C with P(r2i) = 1 such 
that (A.6) holds for all realizations A = A^^ with w € f^i. The law of large 
numbers for the Poisson process (e.g. Asmussen [1, Proposition V.1.4]) shows 
that there exists ^2 C $1 with ^{^2) = 1 such that t~^Ni^{t) — > 1 for all 
a; G ri2- As a consequence, 

t-^N^{Xt) X (A.S) 

for all w S and A > 0. 

Fix an a; € ^li 0,2- Given an arbitrary e > 0, we may then choose a 
to = to{uj, e) such that 



(c-e)+t< / A^{s) ds < {c + e)t 
Jo 
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for all t > to, where we denote n+ = max(ii, 0). The above inequalities 
imply that 

t~^NU{c - e)+t) < t-^N^ (^j^ A^{s) ds^ < t-^NU{c + e)t) 

for all t > to- With the help of (A. 8), the claim follows by letting first t — )• cxd 
and then e 0. □ 
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